Skip to content

Conversation

@ef0xa
Copy link

@ef0xa ef0xa commented Feb 8, 2025

DX-152: runpod-workers/worker-faster_whisper: update for modelcache

Haven't decided yet whether I'll fork this to a new repo entirely or just move this one over, since this ended up pretty much rewriting the whole thing from scratch.

  • leverage the huggingface model cache and get rid of all the bits where we have to do very slow downloads at runtime.
  • got rid of all the overabstracted nonsense, turning this into a handful of simple functions that run from the top to the bottom in a single file
  • make it so that the tests can ever run
  • actually comment the code so the flow isn't obscured
  • update requirements.txt with latest the latest version of runpod
  • try and give helpful error messages to the user!
  • rewrite the entire dockerfile, etc.

@ef0xa ef0xa changed the title fix rphandler & get it ready for modelcache DX-152: fix rphandler & get it ready for modelcache Feb 10, 2025
@ArindamRoy23
Copy link

Can this please be merged. The current code is bad. It loads the model every time, beating the point of faster whisper I guess.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants